Selective Signal Extrapolation and its Application in Image and Video Communications

The estimation of unknown signal samples by extrapolation from known data arises in many signal processing applications. In this thesis, a very generic concept for the estimation of unknown signal samples is derived which is referred to as selective extrapolation. Several applications in image and video communications are discussed from the point of view of extrapolation. Unknown areas occurring in images and videos due to erroneous transmission over unreliable channels are concealed by the proposed selective extrapolation concept. The same technique is applied to the estimation of the video signal covered by TV logos in order to remove the undesired TV logo. Prediction in hybrid video coding is also interpreted as the extrapolation of the already coded samples in order to increase coding efficiency. The derivation of the principle of selective extrapolation relies on arbitrary basis functions which allows for a straightforward adaptation to any given application. Since the generic concept is independent of the dimensions of the considered signal space, the proposed approach can be straightforwardly extended from the two-dimensional derivation to three-dimensions. The new concept of three-dimensional extrapolation allows for an interpretation of a video signal as a three-dimensional volume. Therefore, spatial and temporal correlations of the video signal can be simultaneously used for extrapolation purposes. The interpretation of image and video signals as a linear combination of multidimensional frequencies explains why DFT like basis functions are especially suited in the considered extrapolation context. The application of DFT like basis functions leads to the concept of frequency selective extrapolation. The main advantage of frequency selective extrapolation is the ability to inherently adapt to the local signal characteristics. Image signals with different characteristics such as smooth areas, edges, patterns as well as noise-like areas can be extrapolated. With the novel three-dimensional approach, additionally motion and variations in luminance occurring from frame to frame can be inherently compensated.